Synthesizing face sketches from real photos and its inverse are well studiedproblems and they have many applications in digital forensics andentertainment. However, photo/sketch synthesis remains a challenging problemdue to the fact that photo and sketch have different characteristics. In thiswork, we consider this task as an image-to-image translation problem andexplore the recently popular generative models (GANs) to generate high-qualityrealistic photos from sketches and sketches from photos. Recent methods such asPix2Pix, CycleGAN and DualGAN have shown promising results on image-to-imagetranslation problems and photo-to-sketch synthesis in particular, however, theyare known to have limited abilities in generating high-resolution realisticimages. To this end, we propose a novel synthesis framework called Photo-SketchSynthesis using Multi-Adversarial Networks, (PS\textsuperscript{2}-MAN) thatiteratively generates low resolution to high resolution images in anadversarial way. The hidden layers of the generator are supervised to firstgenerate lower resolution images followed by implicit refinement in the networkto generate higher resolution images. Furthermore, since photo-sketch synthesisis a coupled/paired translation problem where photo-sketch and sketch-photo areequally important, we leverage the pair information in the CycleGAN framework.Evaluation of the proposed method is performed on two datasets: CUHK and CUFSF.Both Image Quality Assessment (IQA) and Photo-Sketch Matching experiments areconducted to demonstrate the superior performance of our framework incomparison to existing state-of-the-art solutions. Additionally, ablationstudies are conducted to verify the effectiveness iterative synthesis andvarious loss functions.
展开▼
